Background: The prediction of prostate motion is important for matching planned and delivered dose distributions in prostate radiotherapy. This study aimed to assess the relationship between anatomical characteristics and inter- and intra-fraction prostate motion. Materials and Methods: Sixty-six patients who underwent fiducial marker implantation were retrospectively evaluated. The anatomical characteristics (subcutaneous adipose tissue thickness, pelvic cavity volume, and fat volume of the lesser pelvis around the prostate), inter- and intra-fraction prostate motion, and standard deviations (SDs) in the anterior-posterior (AP), superior-inferior (SI), and left-right (LR) directions were determined, and their correlations were analyzed. Additionally, the three-dimensional (3D) distance between the coordinates of the center of gravity of the prostate and inferior margin of the symphysis pubis was calculated. Results: The pelvic cavity volume around the prostate exhibited a moderate correlation with the SD for inter-fraction prostate motion in the LR direction (r=0.47) and that for intra-fraction prostate motion in the AP and LR directions (r = 0.41, 0.52). The 3D distance between the coordinates of the center of gravity of the prostate and inferior margin of the symphysis pubis showed a moderate correlation with the average inter-fraction prostate motion in the AP direction (r=0.46). Conclusion: Prostate motion in the AP and LR directions may be related to the fat and pelvic cavity volumes around the prostate. The evaluation of anatomical characteristics can help predict patient-specific prostate motion during treatment planning.